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evaluation, students interviews, and analyses of student performance (pre- 
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using the software, found the constructivist learning environments 
challenging, valued the permanent availability of online information, found 
the user interface of the software products easy to use and navigate. 

Analyses of examination results showed that students performed better than in 
the previous year (traditional lectures) . Results for the carbohydrate course 
were superior to those of the other course. It appears that interactive 
components that foster constructivist-based learning skills are more 
important in online learning environments than presentation of information. 
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Abstract: Development of on-line courses requires the use of appropriate educational 
philosophies that discourage rote learning and passive transfer of information from teacher to 
learner. This paper reports on the development, use and evaluation of two, second year, 
Biology on-line software packages used by students in constructivist environments. The 
courses on carbohydrate and lipid metabolism were developed in conjunction with subject 
experts but were designed from different perspectives. The carbohydrate metabolism course 
provided diverse views of a single knowledge domain and included the ability to find 
information in different ways. The lipid metabolism course was designed as a ‘notes-on-the- 
Web’ module. Evaluations were conducted via paper- and electronic-based software 
evaluation; student interviews; and analyses of student performance (pre- and post-testing; 
examination results). Results showed that students enjoyed using the software, found the 
constructivist learning environments challenging, valued the permanent availability of on-line 
information, found the user interface of the software products easy to use and navigate. 
Analyses of examination results showed that students performed better than in the previous 
year (traditional lectures). Results for the carbohydrate course were superior to those of the 
other course. It appears that interactive components that foster constructivist-based learning 
skills are more important in on-line learning environments than presentation of information. 



Introduction 

Attempts in the 1980s to bring technology into the classroom involved the creation of computer literacy classes 
at secondary school levels (Moersch, 1995) where students learned about computer architecture, operating 
procedures, basic software applications and introductory programming. Technology carries an expectation that it 
will transform many aspects of learning including the content and body of knowledge taught, delivery, and the 
types of facilities required to support delivery (Kook, 1997). Learning environments 1 are becoming more creative 
and diverse with educational institutions becoming, not only information centres for specific content, but also 
arenas for technology development and innovation (Kook, 1997). However, technology is not a process, mindset, 
or a global solution, but is merely a tool. With the onset of the 21st century, it is undeniable that the computer, as 
a tool, has made by far the biggest impact on human endeavour. “Technology, be it a writing system, a media 
system, or a computer system, has been proven to be an effective tool to realize the interrelated goals of learning, 
teaching and cognitive growth” (Chen, 1993). Moersch (1995) stated that computers are used as tools that 
support and extend student understanding of concepts, processes and themes. In addition to its inherent use as 
communication tools, technology can also play a direct role in educational activities. Richards et al (1997) 
argued that the use of technology is two-fold: it accelerates skill and knowledge acquisition and enhances teacher 
and student abilities. McDonald and Ingvarson (1997) argued that computer applications free students from 
laborious tasks of rewriting and copying notes. Dorfler (1991), on the other hand, stated that cognitive software 
tools amplify human ability as “information re -organizational devices” that extend activity and reflection to a 
higher meta-level. 

While the use of computer technology could provide an increasing student population with adequate resources, 
the focus should still lie with educational issues (Hamalainen et aL, 1996) and user needs (Brown, 1995). Collis 
(1997) stressed that technology should be used as a tool to make concrete an educational philosophy and 
Greening (1998) argued that theory should drive the application of technology within educational contexts rather 
than technology itself. Reeves and Hedberg (1998) defined constructivism as “...the process of how we 
construct meaning and knowledge in the world..”, “..based on our previous experiences and how we organize 
those experiences into knowledge structures such as schema and mental modes, and the beliefs that we use to 
interpret the objects and events we encounter in the world”. Constructivism is context dependent rather than 
content dependent (Greening, 1998) and focuses not on the content or its objectives, but on the diverseness and 
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richness of the learning environment and the skills and competencies in learning that may not be directly 
monitored via behaviouristic assessment methods (Reeves, 1992). Today, most commercial educational software 
takes little notice of educational models other than instruction. This presents educational developers with two 
possible options of technology use where either (a) technology is supplemental and rests on existing structures 
(enrichment) or and (b) it is mandatory and changes the course structure completely (re-engineering) (Collis, 
1997). 

This project was based on the concept that integration of software into learning environments will only improve 
learning when courses are re-engineered to support constructivist -learning models and include tools to find, 
explore and organise information. In other words, the power of technology lies not in the presentation of 
information, but rather in the ability of learners to navigate information in different ways to enable their own 
cognitive development. 

Materials and Methods 

Course Design and Development: Two second-year biochemical courses were converted into resource-based 
on-line educational material. The original instructivist courses included notes, diagrams and overhead slides 
developed by the subject experts, based in part on the prescribed textbooks. The re-engineering of these courses 
included the development of a workbook that posed a number of problems, or exercises, supported by on-line 
resources. The first course, carbohydrate metabolism, consisted of a number of pathways and reactions, and 
requires an in depth understanding of compounds involved (their relationships), as well as control and regulation 
systems of the metabolic pathways. Software developed for this course was based on the idea that multiple 
representation of a single knowledge domain would enhance the learning process. Analysis of the lecture course 
identified a number of content building blocks (molecules, reactions, biochemical terminology, processes and 
actual pathways). These knowledge units were individually constructed using Macromedia Flash 2.0 (figures and 
charts) and were stored, with relationship information, in a Microsoft Access database. These building blocks, 
accessed in different ways (using hyperlinks or through a built-in search tool) provided the material for students 
to construct their own representation of the pathways according to problems presented in the workbooks. In 
addition a glossary of terms was provided. The interface (constructed in HTML using Microsoft FrontPage and 
Macromedia Flash 2.0), provided students with a series of notes, which, although static in nature, contained links 
into the database bound knowledge units. Information was also dynamically accessible via the database, where 
objects (terms, molecules, pathways) could be immediately viewed in separate browser windows. The second 
course (describing lipid breakdown) was based on a number of essays broken into separate topics that included 
molecular data. Notes were pre-prepared by the subject expert and converted into HTML (FrontPage 98) and 
graphics and diagrams (Macromedia Flash 2.0). These resources were grouped into essays, structures, processes, 
and transport mechanisms on the introductory page. No search or glossary was provided. 

Learning Environment: Students worked in groups of three or four at a single computer (Intel Pentium 
233MHz, running Microsoft Windows NT, with Internet Explorer 4.0 browsers and Macromedia Flash plug-in 
installed) using the supplied on-line material and textbooks to answer questions posed in workbooks. Assistants 
and the subject expert encouraged group communication and collaboration, provided support and guided learners 
when asked for help. 

Evaluation: A number of different instruments were used to evaluate the courses and included: paper- and 
electronic -based course evaluation; student interviews and analyses of student performance (pre- and post- 
testing; examination results). Paper based evaluations were conducted to measure student attitudes with respect 
to enjoyment of and benefit from the course, types of skills learned, as well as the suitability and feasibility of 
integrating technology into the classroom. Students were asked specific questions pertaining to: content 
structure; environment and learning activities; software usage; course structure; and skills and competencies. 
These evaluations were conducted at the end of the courses and were analysed qualitatively using QSR NUDIST 
(Qualitative Solutions and Research). Students used the on-line Educational Software Evaluation Tool (ESET) to 
assess the design of the user interface, pedagogy and interactivity of the software. These results were further 
analysed using the Wilcoxin Signed Ranks tests (SPSS, SPSS Inc). Thirteen students, who volunteered and 
represented each group, were informally interviewed to probe more deeply student opinions related to the use of 
on-line courseware and their use of the learning resources. Students answered questions in the following 
categories: personal use, perspectives of higher education, the learning environment, and learning outcomes and 
understanding. Interviews were recorded using a tape recorder, transcribed and analysed using QSR NUDIST. 
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Anonymous pre- and post-tests were constructed by subject experts and took the form of short questions (one 
word and short paragraphs) that tested the understanding of the course content. These tests were administered 
prior to, and after, each of the courses. The Mann Whitney tests (SPSS) were used to evaluate these results. 
Performance data (examination results) was analysed using Wilcoxin Signed Ranks (SPSS) tests to identify 
significant differences between computer based and non-computer for the current year. A combination of the 
Mann Whitney and Wilcoxin tests (SPSS) was use to compare results of the two computer based topics with the 
previous year’s results, as well as comparing each of the computer based course performances to the rest of the 



course. 




Category 

Figure 1 . Paper-based student evaluation of the carbohydrate 
metabolism (n-44) and the lipid metabolism (n=33) 
courseware. (Bar - SE; Categories: 1 . Content structure, 2. 
Course structure, 3. Software usage, 4. Environment and 
learning Activities, and 5. Skills and competencies; □ - 
carbohydrate metabolism, □ - lipid metabolism). 
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Figure 2. Student evaluation of (1) User Interface and 
Design and (2) Pedagogy and Interaction using the Education 
Software Evaluation Tool of the carbohydrate metabolism 
(□; n=42) and lipid metabolism (□; n=40) courseware (Bar 
- SE). 



Results 

Conventional paper based evaluations were conducted to determine student opinion with respect to the learning 
environment and use of the software. The questionnaire was divided into six categories (content structure; course 
structure; software usage; environment and learning activities; and skills and competencies) and included 
statements that students ranked (4-point scale) and short paragraphs answers. Students rated both courses in a 
similar manner; the content structure higher than any other category; the carbohydrate course content structure 
higher than that of the lipid metabolism course; and the course structure lower than any other category (Fig. 1). 
For the carbohydrate course the highest rated statements for each category were: content structure - “It was easy 
to find relevant information” (3.2); course structure - “I enjoyed working with Internet-based courseware” (3.2); 
software usage - “I found the glossary very useful” (3.0); environment and learning activities - 
comfortable working with the material presented” (2.8); and skills and competencies - “I enjoyed using the 
software” (3.1). Similarly, for the lipid metabolism course the statements ranked highest per category were: 
content structure - “Development of Internet-based courseware is a good idea” (2.9); course structure 
printed notes should have been provided” (3.1); software usage “I found the visuals very easy to understand” 
(2.8); environment and learning activities - “I felt comfortable” (2.8); and skills and competencies - “The course 
improved my computer skills” (3.1). Statements that scored the lowest included “I would prefer to work on my 
own” (2.2) for both courses, “I found the relevant information easily” (2.3) for the lipid metabolism course, “I 
found the time allocated for computer usage adequate” (2.2) both courses and “The course required a lot of 
problem solving exercises” (2.2) for the lipid metabolism course. 



Using ESET students rated the user interface, design, pedagogy and interaction of the carbohydrate metabolism 
course software higher that the software used in the lipid metabolism course (Fig. 2). For the carbohydrate 
courseware students rated screen appearance (fonts -78.6%; choice of colours - 82.1%), layout (text flow - 
76.2%; consistency - 76.2%), instructions (73.2%); presentation (72%); reading (80.4%); and the search facility 
(100%) highly. For the lipid metabolism courseware students rated screen fonts (71.9%) and colours (72.5%) 
high, and screen consistency (66.3%), picture layout (69.4%), intuitive interface (66.3%) and text flow (61.9%) 
low. Ease of reading was amongst the highest rating components (73.8%) but students found that there were 
insufficient links (59.4%). A Wilcoxin signed ranks test indicated that the User Interface section and Pedagogy 
and Interaction for carbohydrate metabolism software was rated significantly better than the lipid metabolism 
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package (asymptote significance < 0.005, z = -3.845a and asymptote significance = 0.006, z = -2.726a 
respectively). 



Students were interviewed to further probe their opinions regarding the two courses. Eight students (62%) said 
that they have used computers before (between one and two years) and five said that they had not previously 
used computers. Four of the students reported that they had access to computers at home. Those that had not 
used computers before stated that they soon became familiar with the software. When asked if they were aware 
that the computer-based courseware was part of a research project, nine (69.2%) responded positively and four 
said no (30.8%). Those who responded positively felt that the initiative was new and exciting, with them being 
the first to tryout this new form of learning. Students also felt that it was an improvement over conventional 
lecturing since it made learning easier. A few students thought that the approach was experimental and they had 
no reference to previous years and that the results of such initiatives are unpredictable since the use of 
technology is new. Students were asked if the change in learning environment was comfortable to work in. 
Seven students (53.8%) felt that they were comfortable, while three students (23.1%) maintained that they were 
not comfortable at first, but got used to the course after a while. Two students responded that they were initially 
lost and attributed this to their lack of computer knowledge as well as the wealth of information presented. 
However, they reported that they quickly adjusted to the new teaching method. Most of the student comments 
centred around interactions in the classroom and listed being able to work individually and at their own pace as 
an advantage. This was further probed by asking questions related to group work. Positive responses included 
division of labour (two responses), rich social experience (two responses) and ability to discuss problems with 
peers, tutors and the subject expert. Students did however maintain that group interactions often broke the flow 
of concentration, decreased the work rate and this caused them to work individually. Other problems identified 
were a lack of equal contribution, lack of self-discipline and ’'discussion led by one person". Despite these 
problems, seven students indicated that their understanding within the group was greatly improved. The key 
supporting features included: group explanations and discussion of key concepts (five responses); helping one or 
more students with problems and misunderstandings (four responses) and development of shared understanding 
amongst participants (two responses). Students were asked to comment on the difficulty of the content. Eleven 
students (84.6%) regarded the content as simple and easy to understand, even though it may have been difficult 
to work with the software at first. Although the software was well received, the lipid metabolism software was 
found more difficult to use. Students were asked to comment on how they felt using technology compared with 
previous or conventional modes of learning. This question provided the most insightful and varied responses 
amongst students. Students described different ways in which the technology affected them and made many 
references to lectures, which they considered as the conventional mode. Working with software was more fun, 
challenging, much better than lectures, encouraged learning, was interactive, allowed one to find information 
quickly and easily, made one more responsible, built confidence, improved understanding and made 
visualization easier. Students were requested to identify the difference between understanding concepts and 
memorization of content. They clearly understood that there was a difference between understanding and 
memorization, and that understanding took place before memorization. Students maintained that they did not 
remember much of what they learned (five responses), but could understand nearly all of what they covered 
(eight responses). Students also felt that their initial experience with the software mentally prepared them to 
study for examinations. 
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Figure 3. Comparative analysis of average student scores Figure 4. Comparative analysis of student pass rate between 

between two consecutive years (□ - 1998, n=39; □ - 1999, two consecutive years (□ - 1998, n=39; □ - 1999, n=47) 

n=47; bar = SE). 

Pre- and post-tests were used to determine student understanding of the basic course content before and after 
using the developed materials. Students received no prior warning before sitting these test and had therefore not 
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prepared themselves for testing. In the carbohydrate metabolism course, students (n=49) scored an average of 
15.8 ±1.4% that increased to 34.7±1.4% in the post-test (n=44). Using a 95% confidence level, there was a 
significant difference between the pre- and post-test (asymptote significance < 0.005, z =-6.841, Mann- Whitney 
test) indicating that students had gained knowledge during the course. In the lipid metabolism course students 
(n=39) scored an average of 26.1±1.6% for the pre-test and 34.0±2.2% in the post-test (n=30). The Mann- 
Whitney test showed a statistical difference between pre- and post test (asymptote significance = 0.009, z=- 
2.596) indicating that students had gained knowledge during the course. 

Examination results from the previous year (all lecture based) were compared (Fig. 3.). There was a 4.8% 
increase in the current year (66.5%) for the carbohydrate metabolism course and a 5.04% decrease for the lipid 
metabolism courseware (55.5%). Students appeared to perform similarly in both examinations. However, the 
pass rate for the carbohydrate metabolism course increased from 78.4% to 93.6% (1999) while that for the other 
course on lipid decreased from 81.6% to 71.1% (Fig. 4). A comparison (Mann Whitney test) was made between 
1998 examination performance and 1999 exam performance using the following sets of scores (1998 
carbohydrate and lipid metabolism) - (1998 other courses) versus (1999 carbohydrate and lipid metabolism) - 
(1998 other courses). Although no significant difference was indicated (asymptote significance = 0.054, z = - 
19.925), the results can be seen as marginal. The 1999 examination performance results were further analysed 
using the Wilcoxin Signed Ranks test: the carbohydrate metabolism examination results were significantly better 
than either the lipid metabolism (asymptote significance = 0.003, z = -2.948) or the traditional lecture-based 
courses (asymptote significance < 0.005, z = -4.525). No significant difference was found between the lipid 
metabolism course and the traditional lecture course results (asymptote significance = 0.1 1 1, z = -1.594). 

Discussion 

In order to gain insight into the use of on-line material in constructivist classrooms two second-year biochemical 
courses were redesigned. The carbohydrate metabolism course on-line resources included multiple views of the 
knowledge domain, a search tool and glossary. The lipid metabolism course included only text and graphical 
resources on the Web. Both courses used a constructivist approach where students, working in groups, used the 
resources to answer problems posed in workbooks. A number of instruments were used to evaluate the courses 
and included paper and electronic evaluations, interviews and analyses of student performance. 

Jarz et al. (1997) stated that a user interface has a significant influence on the ‘look and feeT of the system and 
its success and includes "symbols and colours" as basic but important components. Barker (1990) stated that one 
of the functions of user interfaces is to enable the learner to visualize what is happening within the learning 
domain with which they are interacting. User interface design should also make full use of technology rather 
than "transferring paper or previous non-graphic interfaces onto the screen" (Starr, 1997). Results from the 
paper- and electronic-based evaluations, as well as student interviews, clearly indicated that students found the 
interfaces easy to use and adapted quickly to the use of the software in the classroom. The carbohydrate 
courseware was more highly rated than that used in the lipid metabolism course. While the user interface of both 
on-line resources was rated highly, the lack of search tools, glossary and self-testing components, which could be 
viewed as cognitive development tools, in the lipid metabolism software resulted in problems in the use of the 
resources and made it more difficult for students to find relevant information. These results support the 
arguments that the inclusion of different navigational tools supports different navigation styles (Jonassen and 
Grabinger, 1990) and learners who are allowed to make choices, answer questions and solve problems, are more 
motivation and productive (Jih and Reeves, 1992). The inclusion of a wider variety of navigational, visualization 
and self-assessment tools in the carbohydrate package supports the idea that software that actively engages 
students in constructive activities biases students towards knowledge transformation activities and away from 
simple knowledge replication (Ward and Tiessen, 1997). 

Gustafson and Branch (1997) argued that the construction of knowledge and skills occur while learners interact 
with peers, media, content and teachers. Berge (1997) introduced the term "authentic learning activities" that 
includes inquiry, problem-based activities, case studies, projects, peer critique and support and self-reflection. 
While some students would have preferred lectures they argued that group work supported the development of 
understanding and felt that group work encouraged learning. 

Reiser and Kegelmann (1994) stated that together with student "attitude data", the pre- and post-test method is a 
way of improving courseware evaluations. Use of such a method here found that students gained some insight 
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into the concepts of both courses. Evaluation of examination results show that students performed better in the 
carbohydrate metabolism course compared to the lipid metabolism and traditionally taught courses. While both 
courses used constructivist principles, performance was only increased in the course that included software that 
provided learners with a richer set of tools to interrogate information and thereby build knowledge. It is 
interesting that students would have preferred written notes for the lipid metabolism course as they are familiar 
and have developed skills in using such media. The carbohydrate metabolism courseware consisted of discrete 
knowledge units (molecules, terms, pathways) and processes (reactions and control aspects) that fit more easily 
into a constructivist model. The lipids metabolism courseware was essay-based and textual in nature. The level 
of interactivity differed significantly between both packages and information was not rigidly structured in the 
carbohydrate on-line resource. Another important feature that distinguished the packages was the ease of 
navigation: the carbohydrate metabolism software was much easier to navigate and far more intuitive than the 
lipid metabolism software. Finally, the carbohydrate metabolism software proved to be more motivating in terms 
of fun, graphics and student interest. The results presented here clearly show that the use of on-line resources to 
provide information only provides no benefit to learners while those that are richly textured and include different 
navigational paths, are built of small interrelated knowledge blocks and include cognitive development tools 
improve learning. Therefore, computer-learning resources that are just electronic textbooks offer little value in 
knowledge development. 
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